Sentinel AI: An AI-Powered Visual Problem Detection and Intelligent Expert Recommendation System Using Computer Vision and Large Language Models

Abstract

Everyday technical failures, ranging from leaking pipes and cracked casings to obscure software exceptions, are difficult for non-specialists to diagnose and even more difficult to resolve without paying for an on-site expert visit. This paper presents Sentinel AI, a multi-role, AI-powered problem-diagnosis and expert-marketplace platform that unifies computer-vision-based hardware fault detection with large-language-model-based software log analysis under a single \"Universal Analysis\" pipeline. The system ingests either a photograph of a damaged physical asset or a snippet of code, logs, or an error trace, classifies the artifact type, estimates severity and confidence, produces a plain-language technical explanation with a recommended remediation, and, where the issue exceeds what a user can safely resolve alone, routes a structured request to the nearest available domain expert through a real-time, Socket.io-based messaging layer with SMS escalation via Twilio. Three cooperating web portals — Customer, Expert, and Admin — are backed by a Node.js/Express service layer and a MongoDB document store. This paper describes the system\'s architecture, the diagnostic and expert-matching pipeline, the technology stack, and the implemented user interfaces, and it discusses the platform\'s advantages, current limitations, and directions for future experimental validation.

Introduction

Sentinel AI is a unified AI-powered diagnostic system designed to identify and resolve both physical hardware faults (e.g., cracked screens, pipe leaks) and software issues (e.g., code errors, logs). Unlike existing systems that only detect problems or are limited to a single domain, Sentinel AI provides end-to-end assistance by diagnosing issues, explaining them in simple language, recommending step-by-step solutions, estimating repair costs, and connecting users with qualified experts when necessary.

Key Problems with Existing Approaches

• Most systems are limited to a single domain (e.g., automotive or infrastructure).
• They detect faults but do not provide actionable solutions.
• They lack human expert support for complex cases.
• Multi-model systems are computationally expensive.
• Limited explainability reduces user trust.

Motivation

The project aims to bridge the gap between fault detection and resolution by offering a single platform that:

• Diagnoses problems from images or text.
• Explains severity and likely causes.
• Suggests practical solutions and cost estimates.
• Connects users with verified experts when AI assistance is insufficient.

Main Contributions

• A Universal Analysis pipeline for both image- and text-based diagnostics.
• Severity and confidence scoring with clear explanations and repair guidance.
• Real-time expert matching using Socket.io, MongoDB, and Twilio SMS.
• A three-role architecture (Customer, Expert, Admin).
• A complete implementation using the Node.js, Express, and MongoDB stack.

Related Work

Previous research has focused on specific areas such as infrastructure monitoring, automotive diagnostics, wildfire detection, and explainable AI. While these systems improve detection or explanation, they:

• Are domain-specific.
• Lack integrated human expert support.
• Do not provide a unified platform for both hardware and software diagnostics.

Sentinel AI addresses these gaps by combining multi-domain diagnostics, AI-powered reasoning, and live expert assistance within a single deployable system.

Problem Statement

The goal is to analyze user-submitted images or software logs and produce:

• Problem classification.
• Confidence score and severity level.
• Technical explanation.
• Step-by-step solution.
• Estimated repair cost.
• Expert matching for unresolved or severe cases.

Proposed Methodology

The system follows a seven-stage pipeline:

1. Data ingestion (images and/or text).
2. Pre-processing (image normalization and text cleaning).
3. Universal problem detection using YOLOv8 for images and language models for text.
4. Contextual aggregation and classification.
5. Solution recommendation.
6. Expert matching and escalation.
7. Output presentation to the user.

Conclusion

This paper presented Sentinel AI, a multi-role platform that unifies computer-vision-based hardware fault detection, LLM-assisted software diagnosis, and a real-time expert marketplace within a single Customer/Expert/Admin architecture. The system\'s Universal Analysis pipeline produces confidence-scored, severity-rated diagnoses paired with actionable remediation and cost estimates, and escalates unresolved or high-severity cases to a verified expert through Socket.io chat and Twilio SMS notification. The working prototype, illustrated through its landing page, diagnostic terminal, and expert portal, demonstrates the feasibility of the proposed architecture on a lightweight Node.js/Express/MongoDB stack. The principal open work is empirical: the accuracy, latency, and user-outcome benefits of the platform have not yet been benchmarked against labeled data or a controlled user study, and Section 8.1 and Section 12 outline the evaluation and extension work required before the system can be considered validated for production deployment. Subject to that validation, Sentinel AI offers a practical template for closing the gap between automated problem detection and human-expert resolution across both hardware and software domains.

References

[1] M. I. Sheikh, \"InfraGPT smart infrastructure: An end-to-end VLM-based framework for detecting and managing urban defects,\" Oct. 2025. [2] I. Nanthini, K. Manivannan, T. Vishnu, S. Thirumalaivasam, M. Yugith, and R. Thanuja, \"Integrating computer vision and deep learning for automated object diagnosis and repair,\" pp. 1–6, May 2025. [3] R. K. Verma, G. Agarwal, N. Pandey, and R. K. Verma, \"Explainable AI for automated defect detection,\" in Advances in Computational Intelligence and Robotics book series, Aug. 2025. [4] A. Ayanzadeh, P. Dixit, S. Kamal, and M. Halem, \"Wildfire VLM: AI-powered analysis for early wildfire detection and risk assessment using satellite imagery,\" Feb. 2026. [5] K. Indrajit, S. M. C, V. Ankit, and P. V. D, \"Explainable artificial intelligence (AI)-based image analytics for automatic damage detection and estimation systems,\" June 2025. [6] [Placeholder — requires verification before submission] Ultralytics, \"YOLOv8 documentation,\" 2023. [7] [Placeholder — requires verification before submission] MongoDB Inc., \"MongoDB and Mongoose ODM documentation,\" 2024.

Copyright

Copyright © 2026 Sanket Kadhao, Abishai Amair, Bhairavi Sawale, Vidhi Menghare, Prof. Priya Khotele . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.